A Systematic Approach to Selecting Maintenance Policies in a Data Warehouse Environment

Most work on data warehousing addresses aspects related to the internal operation of a data warehouse server, such as selection of views to materialise, maintenance of aggregate views and performance of OLAP queries. Issues related to data warehouse maintenance, i.e. how changes to autonomous sources should be detected and propagated to a warehouse, have been addressed in a fragmented manner. Although data propagation policies, source database capabilities, and user requirements have been addressed individually, their co-dependencies and relationships have not been explored. In this paper, we present a comprehensive framework for evaluating data propagation policies against data warehouse requirements and source capabilities. We formalize data warehouse specification along the dimensions of staleness, response time, storage, and computation cost, and classify source databases according to their data propagation capabilities. A detailed cost-model is presented for a representative set of policies. A prototype tool has been developed to allow an exploration of the various trade-offs.